At the tail end of Google's keynote speech at its developer conference Wednesday, Sundar Pichai, Google's CEO mentioned that Google had built its own chip for machine learning jobs that it calls a Tensor Processing Unit, or TPU. The boast was that the TPU offered "an order of magnitude" improvement in the performance per watt for machine learning. Any company building a custom chip for a dedicated workload is worth noting, because building a new processor is a multimillion-dollar effort when you consider hiring a design team, the cost of getting a chip to production and building the hardware and software infrastructure for it. However, Google's achievement with the TPU may not be as earth shattering or innovative as it might seem given the coverage in the press. To understand what Google has done, it's important to understand a bit about how machine learning works and the demands it makes on a processor.

Google Takes Unconventional Route with Homegrown Machine Learning Chips Machine learning actually involves two different computing jobs, the learning and the execution of that learning, which is called inference.